Bringing a health systems modelling approach to complex evaluations: multicountry applications in HIV, TB and malaria.

INTRODUCTION: Understanding how to deliver interventions more effectively is a growing emphasis in Global Health. Simultaneously, health system strengthening is a key component to improving delivery. As a result, it is challenging to evaluate programme implementation while reflecting real-world complexity. We present our experience in using a health systems modelling approach as part of a mixed-methods evaluation and describe applications of these models. METHODS: We developed a framework for how health systems translate financial inputs into health outcomes, with in-country and international experts. We collated available data to measure framework indicators and developed models for malaria in Democratic Republic of the Congo (DRC), and tuberculosis in Guatemala and Senegal using Bayesian structural equation modelling. We conducted several postmodelling analyses: measuring efficiency, assessing bottlenecks, understanding mediation, analysing the cascade of care and measuring subnational effectiveness. RESULTS: The DRC model indicated a strong relationship between shipment of commodities and utilisation thereof. In Guatemala, the strongest model coefficients were more evenly distributed. Results in Senegal varied most, but pathways related to community care had the strongest relationships. In DRC, we used model results to estimate the end-to-end cost of delivering commodities. In Guatemala, we used model results to identify potential bottlenecks and understand mediation. In Senegal, we used model results to identify potential weak links in the cascade of care, and explore subnationally. CONCLUSION: This study demonstrates a complementary modelling approach to traditional evaluation methods. Although these models have limitations, they can be applied in a variety of ways to gain greater insight into implementation and functioning of health service delivery.

An integrated approach to improve maternal and perinatal outcomes in rural Guatemala: A stepped-wedge cluster randomized trial.

OBJECTIVE: To evaluate the impact of an intervention package on maternal and newborn health indicators. METHODS: A randomized stepped-wedge non-blind trial was conducted across six subdistricts within two districts in Guatemala from January 2014 to January 2017. Data on outcomes were collected on all deliveries in all 33 health centers. The intervention package included distribution of promotional materials encouraging health center delivery; education for traditional birth attendants about the importance of health center delivery; and provider capacity building using simulation training. Main outcomes were number of health center deliveries, maternal morbidity, and perinatal morbidity and mortality. RESULTS: Overall, there were 24 412 deliveries. Health center deliveries per 1000 live births showed an overall increase, although after adjustment for secular trends and clustering, the relative risk for the treatment effect was not statistically significant (aRR, 1.04; 95% confidence interval [CI], 0.97-1.11, P=0.242). Although not statistically significant, maternal morbidity (aRR, 0.78; 95% CI, 0.60-1.02; P=0.068) and perinatal morbidity (aRR, 0.84; 95% CI, 0.68-1.05; P=0.133) showed a tendency toward a decrease. CONCLUSION: The present study represents one of the few randomized evaluations of an integrated approach to improve birth outcomes in a low-income setting. ClinicalTrials.gov: NCT0315107.

Ethical issues in the design and conduct of stepped-wedge cluster randomized trials in low-resource settings.

BACKGROUND: Stepped-wedge cluster randomized trials (SW-CRTs) are increasingly popular in health-related research in both high- and low-resource settings. There may be specific ethical issues that researchers face when designing and conducting SW-CRTs in low-resource settings. Knowledge of these issues can help to improve the ethical conduct of SW-CRTs in a global health context. METHODS: We performed an ethical analysis of two studies using SW-CRT designs in low-resource settings: the Que Vivan Las Madres study conducted from 2014 to 2017 in Guatemala and the Atmiyata study conducted from 2017 to 2018 in rural parts of India. For both case studies, we identified and evaluated the classification of the study as research or nonresearch and the ethical issues regarding the justification of the design, including the delayed rollout of an intervention that had a promising effect. RESULTS: In our case studies, some minor ethical issues surfaced about the registration and stakeholder pressure on the order of randomization, but both included good justification for the design and delayed rollout. Our analysis did, however, demonstrate that careful consideration of the role of randomization and registration of the trials is important. DISCUSSION: SW-CRTs can provide an opportunity for rigorous evaluation of interventions destined to be rolled out on the basis of limited evidence. Furthermore, in SW-CRTs, the underlying objective is often to provide a robust evaluation of the effectiveness for generalized dissemination, and this makes the SW-CRT no less a research study than any other form of cluster randomized trial. CONCLUSION: The design and conduct of stepped-wedge cluster randomized trials raises at least two ethical issues that need special consideration in both high- and low-resource settings: the justification for using the design, specifically the delayed rollout of the intervention to the control group, and the classification of the study as research or nonresearch. In our case studies, these issues did not seem to raise special ethical scrutiny in low-resource settings. Further ethical evaluation will hopefully result in specific ethical guidelines for the use of SW-CRTs in both high- and low-resource settings to contribute to responsible functioning of these trials and adequate protection of participants.